Automatic Generation of Building Models Using 2D Maps and Street View Images

We introduce a new approach for generating simple 3D building models by combining building footprints from 2D maps with street level images. The approach works with crowd sourced maps such as the OpenStreetMap and street level images acquired by a calibrated camera mounted on a moving vehicle. Buildings are modeled as boxes extruded from building footprints with the height information estimated from images. Building footprints are elevated in world coordinates and projected onto images. Building heights are estimated by scoring projected footprints based on their alignment with visible building features in images. However, it is challenging to achieve accurate projections due to camera pose errors inherited from external sensors resulting in incorrect height estimation. We derive a solution to precisely locate cameras on maps using correspondence between image features and building footprints. We tightly couple the camera localization and height estimation steps to produce an effective method for 3D building model generation. Experiments using Google Street View images and publicly available map data show the promise of our method.